The present invention relates to a packet generating method when transmitting by multiplexing one program composed of digital image, digital sound, digital data or the like, or multiplexing plural programs, a data multiplexing method using the same, and an apparatus for coding and decoding the transmission data.
This packet multiplexing method is also employed in MPEG (Moving Picture Experts Group), an international standard for data compression, and is designed to compress and encode the image data and sound data, in every frame for image or in every specific number of samples such as 1024 for sound (also called frame in the case of sound in MPEG), and generate a packet called PES packet by assembling one frame or plural frames.
FIG. 19 shows an outline of format of PES packet; a PES packet has a header, and the header includes information such as stream ID showing whether the succeeding data area is image data, sound data, or other data, or trick mode flag showing data for trick playback. It also contains a timing signal showing when to decode and reproduce in the frame unit of image data and sound data.
Each PES packet transmits by further dividing into a plurality of 188-byte long TS packets (transport packets) described below.
FIG. 21 shows an outline of format of TS packet. FIG. 21(a) shows the state of a PES packet divided into plural TS packets to be continuous TS packets.
As shown in FIG. 21(b), a TS packet is divided into a header, an extension header or adaptation field, and a payload for storing the compressed image or sound data.
The header contains, as shown in FIG. 21(c), packet identifier PID, adaptation field control flag, and counter. The PID has a same value when data of same sequence is included in the payload. The adaptation field control flag indicates presence or absence of adaptation field and payload. The TS packet includes both or either one of adaptation field and payload. The counter shows the sequence of TS packets having same PID. The counter value is expressed in four bits, and is increased one by one, returning to o after 15.
FIG. 21(d) is an explanatory diagram of adaptation field, which includes, as shown in the diagram, random access instruction flag showing random access is enabled, discontinuity flag showing the TS packet is discontinuous, and PCR flag showing presence or absence of coding of reference time clock called PCR (program clock reference) necessary when decoding. In MPEG2, the clock frequency of multiplexing apparatus and separating apparatus is specified at 27 MHz, and in order to synchronize the clock of the multiplexing apparatus and separating apparatus, the clock of the multiplexing apparatus is counted at 27 MHz, and the obtained numerical value is sent as PCR. The separating apparatus reproduces the clock synchronized with the multiplexing apparatus from the received PCR. This PCR is not always required to be transmitted, but is transmitted at interval necessary for synchronization of the multiplexing apparatus and separating apparatus, for example, once in about 0.1 sec. Therefore, in the adaptation field, structurally, it may be allowed to be sent or not to be sent as optional field.
Thus constituted image decoder for decoding image data from the packet data is required to detect a special code called sequence start code in the first place. In the TS packet for transmitting image compressed data, when the random access instruction flag is 1, it is clear that the sequence start code is present in the payload.
The discontinuity flag indicates that discontinuity occurs in the value of counter or PCR shown in FIG. 21(c).
This is description about the packet composition for multiplexing, transmitting or recording image data, sound data, or the like of one program, whereas plural programs can be multiplexed in the TS packet row shown in FIG. 21(a). An example of such packet row is shown in FIG. 20. In FIG. 20, aside from the packet of image, sound and data, there is also multiplexed a table packet for identifying each program when plural programs are multiplexed. Each packet contains identifier (PID) and counter. Multiplexing of plural programs is described below.
When selecting and decoding one program from the TS packet row containing plural programs, it is necessary to refer to the tables called PMT (program map table) and PAT (program association table) in order to identify and select plural programs.
The PMT is a table showing the PID of TS packet including image and sound data contained in one program as shown in FIG. 22(a).
The PAT is a table containing the PID of PMT for each program as shown in FIG. 22(b).
The PMT and PAT are multiplexed in the payload of the TS packet.
FIG. 23 shows an example of packet multiplexing apparatus for multiplexing and transmitting by such conventional MPEG2 multiplexing system, and multiplex data decoding apparatus for reproducing data by separating the input multiplex data. First, the operation of a multiplexing unit 1203 is described. A packet row stored in a first memory device 1201 is put into a first buffer 12031 of the multiplexing unit 1203, and a packet row stored in a second memory device 1202 is put into a second buffer 12032. A table packet is stored in a memory 12033.
A CPU 12033 monitors the buffers 12031, 12032, and tilts a multiplex switch 12035 to the buffer side having input of TS packet, and issues the TS packet. The CPU 12033 also multiplexes and transmits the table packet in the TS packet at a specific interval.
The operation of a separating unit 1204 is described below. An input TS packet is first stored in a buffer 12041. A CPU 12042 operates according to the flowchart shown in FIG. 24, and first receives the PAT and extracts the PID of the TS packet including the PMT of the program to be decoded, and then receives the PMT, and extracts the PID of the PS packet including the image and sound data of a selected program. In the input TS packet, a packet including PCR is sent to a clock control unit 1205, and reproduces reference clock necessary for decoding of image and sound signals. The TS packet including image and sound data is issued into decoders 1206, 1207, respectively through a separation switch 12043. In the decoders 1206, 1207, using the reproduced reference clock, image and sound data is decoded and expanded, and the program is reproduced.
The above international MPEG standard is, for example, ISO/IEC JETC1/SC29/WG11, N801, xe2x80x9cISO/IEC 13818-1 International Standard: INFORMATION TECHNOLOGYxe2x80x94GENERIC CODING OF MOVING PICTURES AND ASSOCIATED AUDIO SYSTEMS,xe2x80x9d November 1994.
In such constitution alone, however, various problems occur in the decoding apparatus.
First, in the multiplexing apparatus, if the program is skipped halfway and transmitted, in the decoding apparatus, the counter of the TS packet in FIG. 21(c) is discontinuous, and the decoding apparatus recognizes an error when detecting discontinuity of the counter, and the TS packet is discarded, and normal decoding may be disabled.
Or, if sequence start code is included in the image data, the random access instruction flag may not be always set at 1, and hence the separating unit in FIG. 23 must search the image data in order to judge presence or absence of sequence start code. Therefore, the CPU requires high processing capacity, and the multiplex data decoding apparatus may be expensive.
Thus, when decoding one program, since identifier (flag) for processing securely is not disposed in a proper position in the packet row, data discarding, long processing time, or other inconvenience may occur.
When a packet row is fed by multiplexing plural programs, by referring to the table packet in the procedure shown in FIG. 23 as mentioned above, only the packets composing one program can be selected and decoded. However, considering to change a program in the midst of reception in the multiplex data decoding apparatus, it is necessary to analyze by obtaining PAT and PMT again, and it takes a certain time to change the program and display actually.
The multiplexing frequency of PAT and PMT into packet row is not specified in the MPEG standard. Considering such format as television broadcasting, it is impossible to guess the timing of change of program in the multiplex data decoding apparatus, from the transmission side of packet row. It hence requires to multiplex the tables of PAT and PMT in the packet row at a specific frequency. When the multiplexing frequency of these tables is increased, the time required for acquiring the table can be decreased, but the rate of the table occupied in the packet row becomes high, and the transmission rate increases. To the contrary, when the multiplexing frequency of PAT and PMT is decreased, increase of transmission rate can be avoided, but it takes a longer time to acquire the table.
Incidentally, the PCR is discontinuous at the beginning of a program or when the program is changed over, but it is not guaranteed that the discontinuity flag in FIG. 21(d) is at a position showing discontinuity before a discontinuous packet arrives. Therefore, in a worst case, without discontinuity flag, a next discontinuous PCR may arrive, and the decoding apparatus attempts to be synchronized with a completely different PCR, and the operation becomes unstable.
Moreover, if the frame frequency of image or sampling frequency of audio is changed, the output clock from the decoding apparatus must be also changed, but generally a certain time is needed to establish synchronism, and if receiving data differing in sampling frequency, synchronous system may not be established, and the output may be disturbed.
In this way, in the case of decoding of packet row in which plural programs are multiplexed, since the identifier (flag) for processing securely is not disposed at a proper position in the packet row in one program, unstable operation, long processing time, or other inconvenience is likely to occur.
On the other hand, when multiplexing a TS packet of plural programs, it is possible that the TS packet having reference clock PCR may collide at the time of packet multiplexing, and hence the PCR packet interval may fluctuate to jitter. In the decoding apparatus, the reference clock is reproduced by composing a PLL, but the jitter cannot be absorbed when the PCR interval exceeds a certain value, and reproduction of clock is d isabled.
In FIG. 19, meanwhile, there is a trick mode flag in the PES header, and command for fast forward or freeze can be transmitted together with the PES packet, but the operation of the decoding apparatus is not clearly defined, and secure operation is not guaranteed.
Thus the problem is the operation may be unstable because secure instruction signal for reference clock or trick play is not disposed at optimum position in the packet row.
It is hence a firs t object of the invention to present a packet generating method, a data multiplexing method using the same, and an apparatus for coding and decoding the transmission data, in which a packet row having an identifier disposed optimally for processing, so that the decoding may be processed securely when decoding the packet row, in a packet multiplexing method for transmitting by multiplexing one program composed of digital image, digital sound, and digital data, or multiplexing plural programs.
It is a second object of the invention to present a packet generating method, a data multiplexing method using the same, and an apparatus for coding and decoding the transmission data, in which a packet row having an identifier for processing or data as reference for processing disposed optimally, so that the decoding may be processed at high speed when decoding the packet row, in a packet multiplexing method for transmitting by multiplexing one program composed of digital image, digital sound, and digital data, or multiplexing plural programs.
It is a third object of the invention to present a packet generating method, a data multiplexing method using the same, and an apparatus for coding and decoding the transmission data, in which a secure instruction signal for reference clock or trick play is disposed optimally in the packet row so that the decoding process operation is stable when decoding the packet row, in a packet multiplexing method for transmitting by multiplexing one program composed of digital image, digital sound, and digital data, or multiplexing plural programs.
To achieve the first and second objects, the invention provides a packet generating method, a data multiplexing method using the same, and an apparatus for coding and decoding the transmission data, characterized by generating a packet row of digital data, containing a change identifier for identifying change of digital data in the packet row, and disposing the packet containing this change identifier closest to the packet containing changed digital data, among the packets containing various identifiers in the packet row.
To achieve the first and second objects, moreover, the invention provides a packet generating method, a data multiplexing method using the same, and an apparatus for coding and decoding the transmission data, characterized, when multiplexing, transmitting or recording plural programs, by generating a packet row of digital data including plural programs, containing a change identifier for identifying change of the plural programs in the packet row, and disposing the packet containing this change identifier closest to the packet containing changed program, among the packets containing various identifiers in the packet row.
Further, to achieve the objects, the packet multiplexing method of the invention is characterized by feeding a first packet row containing image, sound or other digital data and identifier for classifying the digital data, feeding a second packet row containing image, sound or other digital data and identifier for classifying the digital data, generating a firstxe2x80x2 packet row by converting the identifier of the first packet row, generating a secondxe2x80x2 packet row by converting the identifier of the second packet row, and multiplexing the firstxe2x80x2 packet row and secondxe2x80x2 packet row to generate and output a third packet row, wherein at least one packet including a xe2x80x9cflag showing discontinuityxe2x80x9d corresponding to a change identifier for identifying the change of discontinuity of digital data is multiplexed and outputted in the firstxe2x80x2 packet row.
At this time, the flag showing discontinuity is to be added to the packet to be multiplexed in the first place in each identifier of the firstxe2x80x2 packet row, or to be added to the packet to be multiplexed in the last place in each identifier of the firstxe2x80x2 packet row, or the packet setting the flag showing discontinuity to 1 is generated and multiplexed before multiplexing the first packet in each identifier of the firstxe2x80x2 packet row, or the packet setting the flag showing discontinuity to 1 is generated and multiplexed after transmitting the last packet in each identifier of the firstxe2x80x2 packet row. Accordingly, if the program sent out from the memory device is skipped halfway and the counter becomes discontinuous in FIG. 21(c), the decoding apparatus can continue the decoding process by referring to the discontinuity flag.
In the packet generating method of the invention, when dividing and generating input digital compressed image data into packets, if a packet contains specific code such as sequence start code and includes first bit of code that can cause change such as start of image sequence, a flag showing that random access is possible (corresponding to change identifier) is added to the packet. Accordingly, if a sequence start code is contained in the packet for transmitting image data, as the flag showing random access is possible is added, it is not necessary to search the sequence start code from the image packet.
Moreover, in the case of generation of multiplex packet by multiplexing a data packet containing at least one kind of digital data, and a table packet containing a table showing the packet content and packet number, when data is changed, such as change of content of table packet, before sending the data packet corresponding to the changed table packet, an instruction signal showing change of table packet (corresponding to change identifier) and changed table packet are multiplexed in the multiplex packet. As a result, when changing the program, the PMT or PCR is sent out prior to the data of changed content, so that the operation of the decoding apparatus may be secure.
Or, the packet of reference time signal to be multiplexed, before the packet containing digital data in the case of multiplexing a packet containing at least one kind of digital data and a packet containing a reference time signal of decoding apparatus, or before the packet containing discontinuous digital data in the case the content of the digital data is discontinuous, adds a reference time signal discontinuity flag showing that the reference time signal is not continuous (corresponding to change identifier), and also multiplexes the flag.
To achieve the third object, the data multiplexing method of the invention is characterized by feeding a first packet row multiplexing a packet containing at least one kind of digital data and a packet containing a reference time signal of decoding apparatus, feeding a second packet row multiplexing a packet containing at least one kind of digital data and a packet containing a reference time signal of decoding apparatus, and multiplexing the first packet row and second packet row to output a third packet row, wherein if a packet containing the reference time signal in the first packet row, and a packet containing the reference time signal in the second packet row are entered within a specific time, the reference time signal is deleted from the packet containing the reference time signal in the first packet row. As a result, the interval of reference signal PCR becomes less than a specific value, and hence clock reproduction is not disturbed.
Also, in the data multiplexing method of the invention, in the case of trick play by using plural packets containing sound data, image data and other digital data, a packet containing image, sound or data composes a trick play packet by data of same frame only, and the trick play packet is transmitted. Accordingly, the data of same picture only is composed into PES packet at the time of trick play and is sent out to the decoding apparatus, so that the image is not interrupted halfway.
In the multiplex data decoding method of the invention, a packet containing digital image data compressed by using compressive coding in frame or field, or compressive coding between frames or fields is decoded, and when freezing in an arbitrary frame or field by using the decoded data, the image decoding the image data by compressive coding in frame or in field immediately before or immediately after is displayed as a still image when receiving a freeze command signal.
Or, in the case of decoding a packet adding a flag showing decoding or display time to the compressed digital image data and freezing in an arbitrary frame or field by using the decoded data, when receiving a freeze command signal, a flag showing decoding or display time of image data that can be accessed immediately before or immediately after in time is stored, and when resuming freezing, the flag showing decoding or display time of the accessible image data is sent to the multiplexing apparatus, and thereby decoding is resumed from the accessible image data. Accordingly, when freezing image display, by freezing in a picture coded in frame and transmitting the time information of the frozen picture to the transmitting device, continuous image can be reproduced after freezing by resending from the image data of frozen picture.
Further, in the multiplex data decoding method of the invention, in a multiplex data decoding apparatus for feeding a packet row in which at least one program is multiplex, and separating into image, sound or other data, if a table packet for identifying the picture, sound or other digital data is multiplexed in each program, the tablet packets other than the programs to be decoded are also stored in the memory. As a result, when recording the digital data composed of packet rows containing plural programs, programs can be changed over promptly by storing other tablet packets than the programs to be decoded into the memory.